Explicit touch selection and cursor placement

ABSTRACT

A system and method for implementing an efficient and easy to user interface for a touch screen device. A cursor may be placed by a user using simple inputs. The device operates places the cursor coarsely and refines the cursor placement upon further input from the user. Text may be selected using a gripper associated with the cursor. The user interface allows text selection without occluding the text being selected with the user&#39;s finger or the gripper. For selecting text in a multi-line block of text, a dynamic safety zone is implemented to simplify text selection for the user.

BACKGROUND

Touch screen devices often have cumbersome on-screen user interfaces.Various ways of interacting with touch screens are known in the art. Forexample, using a stylus pen or fingers as input devices. The userexperience may be further complicated when using one or more fingers asan input device. Placing a cursor and selecting text can be difficultusing a finger because precision is much lower than with other inputdevices, such as a mouse. For example, placing a cursor at a precisepoint within a word can be difficult due to the size of the fingerrelative to the size of the word.

A user may want to place a cursor so that text being displayed by acomputing device may be edited. Similarly, text may be selected so thatit may be copied, cut, or overwritten by pasting text or entering newtext. These operations, which are known in the art, have provendifficult to implement with touch screen devices due to the imprecisionof using one or more fingers to interact with a touch screen. The speedand ease of selecting text is also reduced when the user interfacerequires the user to enter complicated commands such as pressing andholding the selected text. The above operations are an even moredifficult problem for portable electronic devices.

User interfaces known in the art display a cursor that may make itdifficult for a user to discern the exact location where text will beinserted when entered by the user. Furthermore, when selecting text,present user interfaces often require that the user's finger block theportion of text being selected. Thus, these user interfaces oftenutilize an offset representation of the text being selected whichrequires unintuitive and unnecessary hand-eye coordination.

Selecting text on multiple lines can be difficult because the lines oftext typically occupy a small vertical space relative to the size of auser's finger. It is also very difficult for humans to move their fingerin a straight line. This results in errors when a user attempts toselect text on a single line but the user's finger moves just outside ofthe vertical space defined by the line of text causing the computingdevice to interpret the user's input as purposefully changing lines.

SUMMARY

In modern touch screen devices, users expect an intuitive and simpleuser interface that allows efficient operation of the device. Describedherein are techniques for implementing a user interface with simplecursor placement and occlusion-free text selection. The user interfaceis optimized such that users of mobile devices, e.g. handheld devices,laptops, or tablet computers, may quickly and efficiently perform theseoperations.

Cursor placement may be achieved with a simple tap input from a user.Initially, a cursor may be placed coarsely. Upon further input from theuser, the cursor may be placed more precisely. A visual indication of alocation on the screen that the user may interact with, referred to as a“gripper,” may be displayed below the line of text with which it isassociated. The user interface may also implement “safety zones” thatallow the user to more accurately select text on a single line of text

In some embodiments, a cursor may be placed on a display screen of acomputing device by receiving a location indication from a user, whereinthe indication from the user indicates some text or character string. Aninitial cursor location is selected based on the location indication, incombination with other information about the displayed content, and thecursor is displayed at that location, wherein the initial cursorlocation coarsely positioned relative to the location indicated by theuser. The computing device is then placed in a state in which executionof a function is based on the initial cursor location. A second locationindication may then be received. A more precise cursor location may beselected based on the second location indication, and the cursor isdisplayed in the more precise cursor location.

In some embodiments, a computer system with a display screen, a sensorand a processor implement a user interface for selecting text. A stringof characters, such as text, is displayed on the display screen alongwith a “gripper.” A user may drag the gripper, as determined by thesensor, from a first location associated with a first character of thestring to a second character of the string. The text between the firstand second characters is highlighted and a gripper is displayed again atthe first location and a second gripper is displayed at a locationcorresponding to the second character of the string.

In some embodiments, multiple lines of text may be displayed by adisplay screen. A portion of the text may be selected, which may beindicated by highlighting the text. A user may adjust the portion oftext that is selected by dragging an end point of the selected text. Asthe drag input is being received, the selected text is updated based onthe current location the user is indicating. In a first mode, the deviceallows a relatively large threshold for error in the vertical locationindicated by the user such that the end point does not change linesunless the user moves past the threshold distance. Once the thresholddistance is passed and the device continues to receive the drag inputfrom the user, the device enters a second mode where a thresholddistance smaller than the relatively large threshold is used. Thus, inthe second mode, a user can move the end point of the selected text fromline to line by simply passing the relatively small threshold distance.

Some methods for selecting and highlighting may be executed by aprocessor of a computing system executing instructions stored on acomputer readable storage device.

The foregoing is a non-limiting summary of the invention, which isdefined by the attached claims.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings are not intended to be drawn to scale. In thedrawings, each identical or nearly identical component that isillustrated in various figures is represented by a like numeral. Forpurposes of clarity, not every component may be labeled in everydrawing. In the drawings:

FIG. 1 is a block diagram of an exemplary environment in whichembodiments of the invention may operate;

FIG. 2 is a simplified diagram of an exemplary computing device in whichembodiments of the invention may operate;

FIGS. 3A-3C illustrate example apportionments of a string of charactersused in some embodiments of the invention;

FIGS. 4A-4C illustrate example placement of a cursor used in someembodiments of the invention;

FIGS. 5A-5C illustrate example text selection techniques used in someembodiments of the invention;

FIG. 5D illustrates an example context menu used in some embodiments ofthe invention;

FIG. 6A illustrates an example user interface with overlapping grippersthat some embodiments of the present invention avoid;

FIG. 6B illustrates an example user interface of some embodiments of thepresent invention that prevents overlapping grippers;

FIGS. 7A-7D illustrate example multiline text selection techniques usedin some embodiments of the invention;

FIGS. 8A-8E illustrate example grippers used in some embodiments of theinvention;

DETAILED DESCRIPTION

The inventors have recognized and appreciated that to provide an easy touse and efficient user interface for a touch screen device, it isdesirable for placing a cursor to be simple, straightforward and takeinto account the imprecision that results from using a finger to inputcommands to the device. The inventors have further recognized andappreciated that providing a user interface that allows a user tointeract directly with text being selected without a finger occludingthe text results in an intuitive, efficient user experience.

FIG. 1 illustrates an example of a suitable computing system environment100 on which the invention may be implemented. The computing systemenvironment 100 is only one example of a suitable computing environmentand is not intended to suggest any limitation as to the scope of use orfunctionality of the invention. Neither should the computing environment100 be interpreted as having any dependency or requirement relating toany one or combination of components illustrated in the exemplaryoperating environment 100.

Embodiments of the invention are operational with numerous other generalpurpose or special purpose computing system environments orconfigurations. Examples of well-known computing systems, environments,and/or configurations that may be suitable for use with the inventioninclude, but are not limited to, personal computers, server computers,hand-held or laptop devices, cellular phones, tablet computers,netbooks, multiprocessor systems, microprocessor-based systems, set topboxes, programmable consumer electronics, network PCs, minicomputers,mainframe computers, distributed computing environments that include anyof the above systems or devices, and the like.

The computing environment may execute computer-executable instructions,such as program modules. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types.Embodiments of the invention may also be practiced in distributedcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed computing environment, program modules may be located inboth local and remote computer storage media including memory storagedevices.

With reference to FIG. 1, an exemplary system for implementing anefficient, user-friendly user interface includes a general purposecomputing device in the form of a computer 110. Components of computer110 may include, but are not limited to, a processing unit 120 (i.e.processor), a system memory 130, and a system bus 121 that couplesvarious system components including the system memory to the processingunit 120. The system bus 121 may be any of several types of busstructures including a memory bus or memory controller, a peripheralbus, and a local bus using any of a variety of bus architectures. By wayof example, and not limitation, such architectures include IndustryStandard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus,Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA)local bus, and Peripheral Component Interconnect (PCI) bus also known asMezzanine bus.

Computer 110 may include a variety of computer readable media. Computerreadable media can be any available media that can be accessed bycomputer 110 and includes both volatile and nonvolatile media, removableand non-removable media. By way of example, and not limitation, computerreadable media may comprise computer storage media and communicationmedia. Computer storage media includes both volatile and nonvolatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canaccessed by computer 110. Communication media typically embodiescomputer readable instructions, data structures, program modules orother data in a modulated data signal such as a carrier wave or othertransport mechanism and includes any information delivery media. Theterm “modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia includes wired media such as a wired network or direct-wiredconnection, and wireless media such as acoustic, RF, infrared and otherwireless media. Combinations of the any of the above should also beincluded within the scope of computer readable media.

The system memory 130 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 131and random access memory (RAM) 132. A basic input/output system 133(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 110, such as during start-up, istypically stored in ROM 131. RAM 132 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 120. By way of example, and notlimitation, FIG. 1 illustrates operating system (OS) 134, applicationprograms 135, other program modules 136, and program data 137.

The computer 110 may also include other removable/non-removable,volatile/nonvolatile computer storage media. By way of example only,FIG. 1 illustrates a hard disk drive 141 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 151that reads from or writes to a removable, nonvolatile magnetic disk 152,and an optical disk drive 155 that reads from or writes to a removable,nonvolatile optical disk 156 such as a CD ROM, DVD or other opticalmedia. Other removable/non-removable, volatile/nonvolatile computerstorage media that can be used in the exemplary operating environmentinclude, but are not limited to, magnetic tape cassettes, flash memorycards, digital versatile disks, digital video tape, solid state RAM,solid state ROM, and the like. The hard disk drive 141 is typicallyconnected to the system bus 121 through a non-removable memory interfacesuch as interface 140, and magnetic disk drive 151 and optical diskdrive 155 are typically connected to the system bus 121 by a removablememory interface, such as interface 150.

The drives and their associated computer storage media, discussed aboveand illustrated in FIG. 1, provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 110. In FIG. 1, for example, hard disk drive 141 is illustratedas storing operating system 144, application programs 145, other programmodules 146, and program data 147. Note that these components can eitherbe the same as or different from operating system 134, applicationprograms 135, other program modules 136, and program data 137. Operatingsystem 144, application programs 145, other program modules 146, andprogram data 147 are given different numbers here to illustrate that, ata minimum, they are different copies.

A user may enter commands and information into the computer 110 throughinput devices such as a keyboard 162 and a pointing device 161, commonlyreferred to as a mouse, trackball or touch pad. These input devices maybe present in some embodiments, but are not required for operation ofcomputer 110. In some embodiments, a display screen 191 includes a touchscreen sensor 172 that may receive inputs from a user's one or morefingers or other input device, such as a stylus or pen. Other inputdevices (not shown) may include a microphone, joystick, game pad,satellite dish, scanner, or the like. These and other input devices areoften connected to the processing unit 120 through a user inputinterface 160 that is coupled to the system bus, but may be connected byother interface and bus structures, such as a parallel port, game portor a universal serial bus (USB). The monitor 191 or other type ofdisplay device is also connected to the system bus 121 via an interface,such as a video interface 190.

OS 134 may comprise a touch screen engine associated with the touchscreen. The touch screen engine receives input from the touch sensor172, processes the input and provides information pertaining to theinput to other components, such as other components of the OS 134,application programs 135, or other program modules 136. Inputs from thetouch screen sensor 172 may indicate a selection of characters or otheritems displayed on the screen 191. The inputs may also indicate aposition of a cursor, as selected by the user. The touch screen enginemay also receive information from the other components and render theinformation on display screen 191. For example, the OS 132 may provideinformation to the touch screen engine to display in a context menu ondisplay 191. Embodiments of the invention may be implemented to alterthe way components identify selections and cursor locations. Embodimentsmay also alter the way the user interface is presented to indicatecharacter selection and cursor location.

FIG. 2 illustrates a simplified diagram of an exemplary computing device200 of one embodiment of the invention. The computing device 200 mayhave some or all of the components described above in conjunction withFIG. 1, however it is not limited to include all of the features ofFIG. 1. The illustrated components of computing device 200 may be laidout in any manner and are not limited to the specific layout orcomponents shown in FIG. 2.

Computing device 200 comprises a display screen 250 for displaying oneor more strings of characters 260. The strings may comprise anycharacters, for example, letters, numbers, punctuation, and the spacecharacter, to name a few. A string may be a single word, a sentence, aparagraph or any other collection of characters. The example embodimentof FIG. 2, shows the strings of characters 260 comprise a multi-linesentence of text that includes letters, spaces and punctuation. Thoughtext with Latin letters is used as an example of characters herein,embodiments of the invention are not limited to any particularimplementation. For example, some embodiments may use Japanese, Chinese,Korean, Arabic or Hindi characters. Embodiments of the invention may beused with vertical languages, wherein strings of characters are arrangedin the vertical direction.

The display screen 250 of computing device 200 is associated with one ormore touch screen sensors 240. The combination of sensors and displayscreen may be referred to as a touch screen. Thus, the computing device200 may be referred to as a touch screen device. The computing device200 may employ any type of touch screen technology. For example, thetouch screen may be resistive, capacitive, acoustic, infrared or anyother touch screen technology. Input may be received by the touch screensensors 240 by a stylus, pen or a user's body, such as one or morefingers. Embodiments of the invention are not limited to any particularimplementation of touch screens.

In addition to accepting input via the one or more touch screen sensors240, the computing device 200 may have one or more buttons 230 foraccepting input from a user. The buttons may be on the front, back orsides of the computing device 200. They may be mechanical buttons,rotary input devices, capacitive buttons or any other type of inputdevice known in the art. As discussed in connection with FIG. 1, thecomputing device 200 may also have a keyboard, mouse or other inputdevices for receiving input from the user, but these are not necessary.FIG. 2 illustrates buttons 230 as a four-direction directional pad forindicating directional information to the computing device 200. A powerbutton 220 is also shown, which indicates to the computing devicewhether to turn on/off or go into a standby mode. Embodiments of theinvention are not limited to any particular implementation of buttons orinput devices.

A user of computing device 200 may want to perform functions on text 260displayed on the display screen 250. To perform a function, the user mayuse the touch screen to indicate the desired placement of a cursorand/or the desired text to be selected. The cursor may be any visualindicator of a location, such as a caret or an arrow.

Functions may be dependent on the location of a cursor or on text thathas been selected by the user. Functions may also depend on othersetting of the device. For example, if a cursor is placed at aparticular location and additional text is entered by the user, theadditional text may be inserted at the location of the cursor or theadditional text may overwrite existing text following the cursor basedon whether the device is in an insert mode or an overwrite mode. A pastefunction may also be performed based on the placement of the cursor suchthat text that has previously been cut or copied may be inserted at thecursor location. Another example is selecting a delete command, whichmay delete one or more characters immediately adjacent to the cursor,such as characters before the cursor or after the cursor.

The same functions listed above may be executed when text is selected,but the behavior will be different based on text being selected. Forexample, if additional text is entered or pasted when text is selected,the selected text will be overwritten with the additional text.Selecting a delete command will delete the selected text. There may beadditional functions available to the user when text is selected. Forexample, the selected text may be copied or cut. Also, the style of theselected text may be changed by the user. For example, the selected textmay be made bold or italic, the font may be changed, or the size of thefont may be changed.

This different behavior of functions based on whether a cursor is at aspecific location or text is selected may be described as the devicebeing in a first state and a second state.

It should be noted that the computing device 200 may perform correctionsto the user's selections such that the indicated location determined bythe computing device 200 may not correspond exactly to the physicallocation at which a user touch was detected on the touch screen. Forexample, it is known for a user of a touch screen device to touch thescreen at a location that is slightly lower than the actual locationthey wish to indicate. This is merely an issue of perspective that thecomputing device can automatically correct. Thus, embodiments of thepresent invention may user locations that have already been corrected toaccount for this effect and other similar effects. The input received bythe user via the touch screen sensor 240 may be any suitable inputcommand. In some embodiments, the input may be a “tap input,” indicatingthat the user touched the screen for only brief moment. The tap may bedetected by techniques known in the art. A brief moment is typicallyless than one second. In some embodiments, the input may be a draginput, wherein the user touches the screen with a finger at a beginninglocation, creates a path by dragging the finger across the screen (whilemaintaining contact with the screen), and terminates the path by liftingthe finger at an ending location. In other embodiments, the input may bea press and hold input, wherein the user touches the screen at alocation, holds it for a period of time, and then terminates contactwith the screen. Each of these types of input may be detected usingdifferent techniques as is known in the art. Embodiments of theinvention are not limited to any particular type of input.

Due to the size of a user's finger, which may be, for example 1-2 cm inwidth or height, location indications received via a touch screen may beimprecise relative to items displayed on the screen (fractions of 1 mmin size). Therefore, the user interface of some embodiments of theinvention will, upon a first location indication from a user, place thecursor in an approximate location relative to the string of charactersbeing selected. Then, if the user wishes to place the cursor moreprecisely, a second indication may be input to the device and the cursorwill be placed at a more precise location associated with the input.This approximate, or rough, placement of the cursor may be implementedin any way and embodiments of the invention are not limited in thisrespect. If the computing device places the cursor using approximateplacement, the computing device may be said to be in a first state. Ifthe computing device is placing the cursor using more precisely, thecomputing device may be said to be in a second state. In someembodiments, whether the device will use precise placement orapproximate placement will depend on characteristics of the objectsbeing displayed. For example, if the display includes text that islarger than a predetermined threshold, then the device may onlyimplement precise placement of the cursor. In some embodiments, thepredetermined text size threshold may be related to the approximate sizeof a user's finger. Thus, the device may not use approximate placementwhen the size of the user's finger is approximately the same size as theitems displayed on the screen.

Approximate placement of a cursor 420 may be implemented in any suitableway. In some embodiments, an input is received from a user thatindicates a particular string of characters on the display screen. Thecomputing device 200 determines the input to be a command to place thecursor at a location associated with that word. The number of possiblelocations at which a cursor may be placed may be reduced compared towhen precise placement is being used. Fewer possible locations resultsin course cursor placement, whereas fine cursor placement has a largernumber of possible locations. For example, approximate cursor placementmay only allow placement of the cursor at the beginning of a string 300,as shown in FIG. 4A, or the end of the string, as shown in FIG. 4B.Determining whether the cursor is placed at the beginning or the end ofthe string may be determined in any suitable way. For example, the wordmay be split into multi-character portions, such as a first half and asecond half. If the location of the user's input indicates the firsthalf, then the cursor will be displayed at the beginning of the string.If, on the other hand, the user's input indicates the second half, thenthe cursor will be displayed at the end of the string. This type ofapproximate placement accounts for the fact that a user may want toplace the cursor at the beginning or end of a word, rather thansomewhere in the middle, even if the user's input does not indicate alocation at the beginning or end of a word. Thus, approximate placementmakes it easy for a user to specify the beginning or end of a word, butprovide an easy way to switch to a more precise cursor placement mode.

In the above example where the beginning and the end of a string are theonly options for approximately placing the cursor, the computing device200 may determine whether to place the cursor at the beginning or theend in any suitable way. In some embodiments, as illustrated in FIGS. 3Aand 3B, the string 300 is split into two portions. If the user's inputindicates the beginning portion of the string, then the cursor 420 isplaced at the beginning of the string, i.e. before the beginningcharacter 310 of the string 300. If the user's input indicates theending portion of the string, then the cursor 420 is placed at the endof the string, i.e. after the ending character 320 of the string.

The string may be split into portions in any suitable manner. Forexample, FIG. 3A illustrates the string “Computer” split in half suchthat there are an equal number of letters in each portion. The firstportion 330 comprises “Comp” and the second portion 340 comprises“uter.” Alternatively, FIG. 3B illustrates the same string 300 split inhalf such that there are an equal number of pixels in each half. Thefirst portion 350 comprises “Com” and a small portion of the letter “p”and the second portion 360 comprises the remaining portion of the letter“p” and the letters “uter.” Note that the size of the portions differdepending on how the string is split. Embodiments of the invention arenot limited to splitting strings into equal portions. For example, theending portion may be intentionally larger than the beginning portion ifthe designer of the user interface believes that a user is more likelyto want the cursor placed at the end of a string rather than at thebeginning of the string. Embodiments of the invention are not limited toa particular implementation of segmentation.

Furthermore, embodiments of the invention are not limited to splittingthe string 300 into two portions. For example, in FIG. 3C the string 300is split into three portions: a beginning portion 370 comprises thefirst three characters of string 300; an ending portion 380 comprisesthe last three characters of string 300; and an intermediate portion 390comprises the remaining characters of string 300, between the beginningportion 370 and the ending portion 380. One of skill in the art wouldrealize that any predetermined number of characters may be used todefine the beginning and ending portions. The illustration of FIG. 3Cused three characters by way of example.

If the user's input indicates a location corresponding to the beginningportion 370 of string 300, then the cursor 420 will be placed before thebeginning letter 310 of the string 300, as shown in FIG. 4A. If theuser's input indicates a location corresponding to the ending portion380 of string 300, then the cursor 420 will be placed after the endingletter 310 of the string 300, as shown in FIG. 4B. If the user's inputindicates a location corresponding to the intermediate portion 390 ofstring 300, then the cursor may be placed in any suitable way. In someembodiments, the cursor may be placed at the beginning of theintermediate portion 390. In other embodiments, the computing deviceplaces the cursor at the end of the intermediate portion 390. Someembodiments may place the cursor in the middle of the intermediateportion. In yet another embodiment, when a user's input corresponds tothe intermediate portion 390, the computing device 200 may place thecursor using precise placement such that the cursor is placed at alocation corresponding to the location indicated by the user's input.For example, in FIG. 3C, if the user indicates a location between the“p” and “u” of the string, then the cursor will be placed at thatlocation, as shown in FIG. 4C. Whereas if the user indicates a locationnear the beginning of the “p,” the cursor will be placed before the “p.”Embodiments of the invention are not limited to a particularimplementation of cursor placement.

Once computing device 200 coarsely places the cursor 420, the user maywish to place the cursor 420 at a more precise location within the samestring 300. The number of possible locations to place the cursor isgreater than the number of locations available when using coarseplacement. Precise placement, or fine placement, of a cursor 420 may beimplemented in any suitable way. For example, if the user indicates asecond location within the same string 300, then the computing device200 will select a location corresponding to the second location from aplurality of possible placement locations, wherein there is a greaternumber of possible placement locations when the computing device 200performs precise cursor placement than the number of possible placementlocations that were available during approximate cursor placement. Insome embodiments, the plurality of possible placement locations maycomprise each location adjacent to each character of the string 300. Forexample, a user's first input may correspond to a first location and thecursor may be approximately placed at the beginning of the string 300(see FIG. 4A). When a second input is received corresponding to a secondlocation between the letter “p” and “u”, then the computing device willplace the cursor 420 between the “p” and “u,” as shown in FIG. 4C.

In some embodiments, both the first input, for approximate cursorplacement, and the second input, for precise cursor placement, may betap inputs. This allows the user to quickly place the cursor at adesired location very quickly and accurately without relying on inputssuch as a drag input or a press and hold input. Tap inputs areparticularly advantageous in portable devices. Embodiments of theinvention are not limited to the type of input the user uses.

In some embodiments, when the display screen 250 displays cursor 420, a“gripper” 410 is also displayed. A gripper is a graphical indication onthe screen with which a user may interact and convey further inputinformation to the computing device 200. A gripper may be implemented inany suitable way. For example, the gripper 410 may be displayed belowthe cursor 420, as illustrated in FIGS. 4A-4C. Embodiments of theinvention are not limited to any particular size of gripper. Forexample, grippers may be in the range of 5 mm to 15 mm in size. In someembodiments the grippers are 8 mm wide. It is possible that the grippersmay be larger in one dimension than in a second dimension. For example,a gripper may be elliptical, having a height of 10 mm and a width of 6mm. The exact size of the gripper is not crucial and any size may beused. In some embodiments the size of the gripper is independent of thescreen size and resolution. For example, a gripper displayed on thescreen of a mobile phone device may have the same width as a gripperdisplayed on a large touch screen display screen associated with acomputer. As a result, when rendered, parameters used by computingdevice 200 to render the grippers may be selected to yield the selectedsize. These parameters may include the screen height, screen width,pixel size, and/or screen resolution.

FIGS. 8A-8E illustrate example embodiments of grippers. As will bediscussed in more detail below, when text is selected by a user, it maybe highlighted and two grippers may be displayed: one gripper at thebeginning of the selected text and one gripper at the end of theselected text. FIG. 8A shows an embodiment wherein the first gripper 810and the second gripper 820 are identical circles. The interior of thecircles may be a solid color, completely transparent, or any oftransparency between a solid color and completely transparent.

FIG. 8B shows example grippers 820 and 822 that are circles with agraphic on the interior. Any suitable graphic may be used. In theexample shown, vertical lines are used to indicate texture such that auser will realize that the grippers are interactive and may be used toinput commands.

FIG. 8C illustrates a beginning gripper 830 that is a mirror image ofthe ending gripper 832. The two grippers are triangles that pointoutward. Like with the circular gripper, the interior of the trianglesmay be fully opaque, fully transparent, or anywhere in between. FIG. 8Dalso shows triangular grippers 840 and 842, but with the addition ofvertical lines to more precisely indicate the where the beginning andend of the selected text is.

FIG. 8E illustrates grippers 850 and 852 that are non-standard shapes.They are rounded at the bottom with a point near the top that can moreprecisely indicate the beginning and end of the selected text than acompletely round shape.

The aforementioned grippers are for exemplary purposes and embodimentsof the invention are not limited to any particular shape. For example,any image or icon may be selected by the user for the gripper image. Inthis way, grippers may be personalized to the user's preferredexperience. In some embodiments the grippers are always displayed belowthe line of text associated with the selected characters. In the casethat text is not selected and, instead, a cursor is displayed, a grippermay be displayed below the cursor. Further, one of ordinary skill in theart would realize that there are many other variations of gripper shapeand placement not shown that may be used and are covered by embodimentsof the invention claimed herein. For example, if a vertical language isbeing displayed, grippers may be displayed to the left of the text.

As discussed above, the computing device 110 may receive input fromvarious devices other than the touch screen sensor 172. In someembodiments, if the cursor 420 is being placed using these otherdevices, the touch screen engine may not display gripper 410 on displayscreen 191. For example, if the arrow keys on keyboard 162 or buttons230 are used to move the cursor, the gripper may not be displayed.

Using grippers to select text will now be discussed in conjunction withFIGS. 5A-5C. FIG. 5A illustrates the same cursor 420 location that wasillustrated in FIG. 4B. The cursor 420 may be placed after the endingletter 320 of string 300 in any suitable way, for example, byapproximate cursor placement. Computing device 200 displays gripper 410below the cursor 420 such that when a user selects the gripper with afinger, the text associated with the cursor 420 and the gripper 410 isnot occluded by the gripper or the user's finger. A user may press anddrag the gripper to implement a “drag input.” In some embodiments, whenthe touch screen engine receives a drag input from the user, but theinitial location of the drag input does not correspond to the gripper,text will scroll or pan to display additional text, when it isavailable.

FIG. 5B illustrates a path 525 that a user's finger may trace acrossdisplay screen 250 in a direction along the line of text. In thisexample, the path 525 begins at a location corresponding to gripper 410,continues to the left, along the direction of the line of text, andultimately terminates at a location denoted by an “x” when the userlifts the finger off the screen 250. Because the gripper 410 is farenough below the text so as not to visually block the text, the path maybe below the string such that while the text is being selected, the usercan still maintain visual line of sight with the text.

In some embodiments, the characters of the string are highlighted as thepath 525, indicating a selection, is received from the user. Forexample, as the user's finger follows the path beneath the letter “r,”the highlighted portion will grow to encompass the letter. Thehighlighting may be a shading 520 of the background behind thecharacters being selected and may indicate to the user that thecharacter is being selected. As the user's finger continues, the otherletters traversed by path 525 become part of the highlighted portionuntil the user completes the drag input by lifting the finger from thedisplay screen 250. The ending location of the drag input corresponds toa character 530 and the text between character 530 and character 320,corresponding to the original cursor location, will be selected. Asnoted above, because the path is below the string, the user may maintainvisual line of sight with the characters being selected.

In some embodiments the touch screen engine will not display gripper 410and/or the cursor 420 while the drag input is being entered by the user.This removes unnecessary graphics that may distract or confuse the userand allows the user to simply concentrate on the characters beinghighlighted.

FIG. 5C illustrates the selected text after the drag input is complete.The touch screen engine instructs the display screen 191 to highlightthe text between letter 320, corresponding to the original location ofthe cursor 420, and letter 530 corresponding to the location where path525 ended. Gripper 410 may be displayed at the same location as beforethe drag input was received. The touch screen engine displays a newgripper 510 at the beginning of the selected characters of the string300. The selected text may be further changed by similar interactionswith either gripper 510 or gripper 410. A user may select more or fewercharacters by providing further drag inputs with starting locationscorresponding to one of the grippers.

In some embodiments, touch screen sensor 172 may receive a drag inputfrom the user corresponding to gripper 410, shown in FIG. 5C. The draginput may terminate on the other side of gripper 510. For example, ifthe drag input terminated at the character “C” of string 300, then uponrelease of the drag input, the touch screen engine selects thecharacters “Comp”, i.e. the text between the location of gripper 510when the drag input was initiated and the location corresponding to theending point of the drag input.

In some embodiments, the touch screen engine changes the shape and/orlocation of the grippers so that the two grippers do not overlap. FIG.6A illustrates a problem that exists if grippers are allowed to overlap.This situation may arise when the number of characters is small,resulting in the width of the selected text to be less than twice thewidth of the grippers. This may also occur when the screen size issmall, such as on a handheld device, and the grippers are set to befixed size regardless of the screen size and resolution. Gripper 610 isassociated with the beginning of the selected character of string 600and gripper 620 is associated with the end of the selected character.The selected character is highlighted 630, as discussed above. Theoverlap between gripper 610 and gripper 620 will make selecting aparticular gripper difficult for the user. The overlap may also bevisually confusing to the user. FIG. 6B shows the result of keeping thegrippers the same size, but moving the grippers away from the ends ofthe selected text. Both gripper 610 and gripper 620 are displayedfurther away from the highlighted text than they are when a greaternumber of characters is selected, for example in FIG. 5C. This preventsoverlapping grippers and allows the user to easily interact with eachgripper individually without confusing the user.

Once the user selects text and the touch screen engine highlights theselected text, as illustrated in FIG. 5C, the user may select a functionto execute based on the selected portion of text. As discussed above,examples of functions may be overwriting the text, re-formatting thetext, deleting the text, or copying the text. Another function that maybe performed based on the selected text is display a context menu 550.The context menu 550 may be displayed, for example, by providing aninput corresponding to the highlighted text or one of the two grippers410 and 510 as illustrated in FIG. 5D. The input may be in the form of atap input, for example. The context menu 550 provides the user withvarious additional functions that may be performed to the selected text.The function options may be provided to the touch screen engine from theOS 134 or an application program 135. For example, if a web browserapplication is providing the text information to the touch screen enginefor display on the screen, then it may be the web browser that providesthe information for the context menu. The various additional functionprovided by the context menu 550 may be, for example, options to cut orcopy the selected characters. The functions could also include optionsfor re-formatting the selected text. The context menu 550 may also offeran option to paste characters that have been previously copied andoverwrite the selected text. The context menu may provide many otheroptions, which are not described herein, but one of ordinary skill inthe art would understand to be covered by embodiments of the inventionclaimed herein. Furthermore, the context menu may be of any shape orsize. For example, a context menu may be a rectangular menu, withfunctions displayed in a vertical list, as illustrated in FIG. 5D. Insome embodiments, the context menu may be circular or elliptical anddisplay functions radially. Embodiments of the invention are not limitedto any particular size or shape of a context menu, nor are they limitedto providing any of the specific functions described above.

Similarly, even when text is not selected and only a single gripper isdisplayed, as illustrated in FIG. 5A, a tap input of gripper 410 mayresult in the display of a context menu. The context menu may notdisplay cut or copy options, since no text is selected, but it mayprovide a paste function or any other suitable function.

The above discussion of character selection was limited to text on asingle line. However, selecting a subset of characters from a text blockthat occupies multiple lines is also an aspect of some embodiments ofthe invention. Specifically, the ability to determine when a userintends to select text on a different line than the line of textassociated with the initiation of a drag input from a user. Input from auser's finger is imprecise and humans tend to trace paths on touchscreens that are not straight and deviate from the user's intended path.Thus, some embodiments of the invention provide a “safety zone”associated with each line of text. A safety zone is a region surroundinga line of text wherein, as long as the path associated with a drag inputfrom a user remains within the safety zone, the computing device 200will determine that the user intended for the selection to stay on thesame line of text. Safety zones also ensure that a selection initiatedby the user begins on the line with which the cursor was originallyassociated. In some embodiments, the threshold distance away from a lineof text that defines the boundary of a safety zone may change based onthe actions of the user.

As an example of some embodiments of the invention, FIG. 7A shows text700 that occupies a top line 710, a middle line 720, and a bottom line730. Each line extends in one direction, for example, horizontally,while each line is adjacent to the next line in a second direction, forexample, vertically. The cursor 712 is located at the beginning of theword “quick” on the first line 710. The cursor may have been placed atthat particular location using any of the cursor placement techniquesdescribed above. The touch screen engine displays a gripper 714 belowthe cursor 712.

A user may enter a drag input beginning at the gripper 714, following apath 716, which terminates in a location associated with the second line720. A safety zone boundary 740 is illustrated by a dashed line in FIG.7A, though the line may not be displayed on the display screen. Theboundary 740 is a certain distance from the first line of text 710 withwhich it is associated. The distance may be a function of any number ofparameters. For example, the distance may be some predetermined lengthaway from the text or a predetermined number of pixels away from thetext. In some embodiments, the distance may be dynamic. The distance maybe a function of the vertical line height, the size of the space betweenlines of text, the screen size, or the screen resolution. In the exampleshown in FIG. 7A, the boundary 740 is at a vertical level occupied bythe second line of text 720. If the path 716 of the drag input does notpass the boundary 740, the computing device determines that the user didnot intend to select text in the second line 720. For example, the path716 illustrated in FIG. 7A terminates just above the boundary 740 and ata horizontal location corresponding to a location between the letters“o” and “w” on the first line of text 710. Therefore, the computingdevice determines that the user wanted to select the text “quick bro” onthe first line 710.

FIG. 7B illustrates what would be displayed after receiving the abovedrag input corresponding to path 716 of FIG. 7A. The touch screen enginehighlights 722 the text “quick bro” and there is a first gripper 714 atthe beginning of the highlighted text and a second gripper 724 at theend of the highlighted text. The user may enter a second drag input witha starting point corresponding to gripper 724 and following path 726,which terminates between the “o” and “v” of the second line of text 720and is below the boundary 740. Because the drag input terminated belowthe boundary 740, the computing device 200 determines that the userintended to select text in the second line 720. Therefore, FIG. 7C showsthat the all of the characters after the location associated with thefirst gripper 714 on the first line are selected and all of thecharacters from the beginning of the second line to the locationassociated with the termination of path 726 is highlighted 722.

FIG. 7C also illustrates a change in the threshold distance associatedwith the safety zone of the first line 710. While the user enters thedrag input, if the path 726 crosses the boundary 740 associated with thesafety zone of the first line 710, then the location of the boundarywill shift to be close to the text of the first line, as illustrated byboundary 741. By giving the user leeway in the location of the pathwhile selecting text, it is easier for the user to select text only on asingle line until the user's input indicates that the user wants toexpand the selected text to include adjacent lines in the block of text.Using the boundary 740 that is farther away is associated with a firstmode of operation. Once the user's intention to select text on adifferent line is made clear by passing over the threshold distanceassociated with the boundary 740, the computing device makes changingthe end point of the selected text easier by reducing the thresholddistance to some value less than the original threshold distance. Thisreduced distance boundary is associated with a second mode of operation.

The second mode is implemented after the initial boundary 740 istraversed and while the drag input is still being input by the user.When the computing device 200 operates in the second mode, each line hasa safety zone boundary that is a shorter distance away from the line oftext than when the computing device 200 operates in the first mode. Forexample, the safety zone boundary could have the same reduced distancethreshold has boundary 741. FIG. 7C illustrates a bottom boundary 742associated with the safety zone of the second line of text 720.

A user may enter a third drag input using the touch screen of computingdevice 200. FIG. 7C illustrates a path 736 associated with this draginput, which begins at the gripper 714 associated with the beginning ofthe selected text. As discussed above, while in the first mode, thethreshold distance to change lines is relatively far away from the lineof text and illustrated by boundary 740. Once the path 736 crossesboundary 740, the computing device enters the second mode, wherein thethreshold is reduced and represented by boundary 741. The safety zoneboundary of the second mode associated with the second line of text 720is represented by boundary 742. The path 736 terminates at a locationassociated with the space between the letter “l” and the letter “a” onthe third line 730. As illustrated in FIG. 7D, the gripper 714, whichwas initially associated with the beginning of the selected text is,after the third drag input, associated with the end of the selectedtext. Similarly, the gripper 724, which was originally associated withthe end of the selected text is, after the third drag input, associatedwith the beginning of the selected text.

In some embodiments, as drag inputs are received from the user, thehighlighting 722 is updated based on the current location along the pathassociated with each drag input. In some embodiments, the grippers 714and 724 may not be displayed while the drag input is being received.

The safety zones discussed above do not apply to the direction in whichthe lines of text are oriented. For example, in the above discussion,safety zones only applied to the direction perpendicular to the lines oftext, e.g. the vertical direction. Determining the end points of thehighlighted portion of text along the direction of the line of text,e.g. the horizontal direction may be done the same way as described inconjunction with FIGS. 5A-5C. Along this horizontal direction, thehighlighted portion of text may be updated one character at a time asthe user's drag input passes the characters.

Having thus described several aspects of at least one embodiment of thisinvention, it is to be appreciated that various alterations,modifications, and improvements will readily occur to those skilled inthe art.

Such alterations, modifications, and improvements are intended to bepart of this disclosure, and are intended to be within the spirit andscope of the invention. Further, though advantages of the presentinvention are indicated, it should be appreciated that not everyembodiment of the invention will include every described advantage. Someembodiments may not implement any features described as advantageousherein and in some instances. Accordingly, the foregoing description anddrawings are by way of example only.

The above-described embodiments of the present invention can beimplemented in any of numerous ways. For example, the embodiments may beimplemented using hardware, software or a combination thereof. Whenimplemented in software, the software code can be executed on anysuitable processor or collection of processors, whether provided in asingle computer or distributed among multiple computers. Such processorsmay be implemented as integrated circuits, with one or more processorsin an integrated circuit component. Though, a processor may beimplemented using circuitry in any suitable format.

The various methods or processes outlined herein may be coded assoftware that is executable on one or more processors that employ anyone of a variety of operating systems or platforms. Additionally, suchsoftware may be written using any of a number of suitable programminglanguages and/or programming or scripting tools, and also may becompiled as executable machine language code or intermediate code thatis executed on a framework or virtual machine.

The terms “program” or “software” are used herein in a generic sense torefer to any type of computer code or set of computer-executableinstructions that can be employed to program a computer or otherprocessor to implement various aspects of the present invention asdiscussed above. Additionally, it should be appreciated that accordingto one aspect of this embodiment, one or more computer programs thatwhen executed perform methods of the present invention need not resideon a single computer or processor, but may be distributed in a modularfashion amongst a number of different computers or processors toimplement various aspects of the present invention.

Computer-executable instructions may be in many forms, such as programmodules, executed by one or more computers or other devices. Generally,program modules include routines, programs, objects, components, datastructures, etc. that perform particular tasks or implement particularabstract data types. Typically the functionality of the program modulesmay be combined or distributed as desired in various embodiments.

Also, data structures may be stored in computer-readable media in anysuitable form. For simplicity of illustration, data structures may beshown to have fields that are related through location in the datastructure. Such relationships may likewise be achieved by assigningstorage for the fields with locations in a computer-readable medium thatconveys relationship between the fields. However, any suitable mechanismmay be used to establish a relationship between information in fields ofa data structure, including through the use of pointers, tags or othermechanisms that establish relationship between data elements.

Various aspects of the present invention may be used alone, incombination, or in a variety of arrangements not specifically discussedin the embodiments described in the foregoing and is therefore notlimited in its application to the details and arrangement of componentsset forth in the foregoing description or illustrated in the drawings.For example, aspects described in one embodiment may be combined in anymanner with aspects described in other embodiments.

Also, the invention may be embodied as a method, of which an example hasbeen provided. The acts performed as part of the method may be orderedin any suitable way. Accordingly, embodiments may be constructed inwhich acts are performed in an order different than illustrated, whichmay include performing some acts simultaneously, even though shown assequential acts in illustrative embodiments.

For example, one embodiment disclosed above is directed to approximateand precise placement of a cursor within a string of text. Anotherembodiment is directed to selection of a string of text using grippers.These embodiments may be combined such that rough and precise placementare used while performing text selection for a gripper. For example, theending location of a drag input from a user may initially be determinedusing approximate placement. Then, upon further input from the user, thegripper may be placed more precisely. Any suitable input from the usermay be received. For example, the user may provide a tap input withinthe selected string of text or provide an additional drag inputassociated with the gripper to precisely place the gripper.

Use of ordinal terms such as “first,” “second,” “third,” etc., in theclaims to modify a claim element does not by itself connote anypriority, precedence, or order of one claim element over another or thetemporal order in which acts of a method are performed, but are usedmerely as labels to distinguish one claim element having a certain namefrom another element having a same name (but for use of the ordinalterm) to distinguish the claim elements.

Also, the phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including,” “comprising,” “having,” “containing,” “involving,” andvariations thereof herein, is meant to encompass the items listedthereafter and equivalents thereof as well as additional items.

What is claimed is:
 1. A method for placing a cursor on a display screenof a computing device displaying a string of characters comprising aplurality of lines of text, each line of text extending in a firstdirection and the plurality of lines of text being offset from oneanother in a second direction, the method comprising: displaying aportion of the plurality of lines of text as selected text, the portionhaving a beginning location and an ending location; and based at leaston user input indicating updates to the ending location of the selectedtext, wherein an initial line includes the ending location prior to theupdates and wherein a current line is initially the initial line,interactively adjusting the portion of the plurality of lines of textdisplayed as the selected text while updates to the ending location arestill being received, by: updating the ending location along the currentline of the text based on a position in the first direction indicated bythe user input; and updating the ending location based on a position inthe second direction indicated by the user input, updating the endinglocation in the second direction comprising: while the computing deviceis operating in a first mode and the user input indicates an updatedending location that is less than a first threshold distance away fromthe initial line in the second direction: maintaining the endinglocation in the current line; while the computing device is operating inthe first mode and the user input indicates an updated ending locationthat is greater than the first threshold distance away from the initialline in the second direction: changing the ending location to anadjacent line, the adjacent line being adjacent to the initial line andoffset in the second direction, and entering a second mode after settingthe current line to be the adjacent line; and while the computing deviceis operating in the second mode, using a second threshold distance inthe second direction from the current line, wherein the second thresholddistance is less than the first threshold distance, to determine whetherto change the ending location of the selected text to a different linefrom the current line.
 2. The method of claim 1, wherein the displayscreen is a touch screen, and wherein the method further comprises:displaying a first gripper at a first gripper location associated withthe beginning location; and displaying a second gripper at a secondgripper location associated with the ending location.
 3. The method ofclaim 2, wherein displaying the first gripper and the second grippercomprises: determining a width of the first gripper and the secondgripper based on display screen parameters.
 4. The method of claim 2,wherein: when the beginning location and the ending location are closertogether than a predetermined distance associated with a width of thefirst gripper and the second gripper, the second gripper location is adistance from the first gripper location such that the first gripper andthe second gripper do not overlap.
 5. At least one storage device havinginstructions recorded thereon which, when executed by a processor of acomputing device comprising a display screen that displays textcomprising a plurality of lines of text, each line of text extending ina first direction and the plurality of lines of text being offset fromone another in a second direction, perform a method comprising:displaying a portion of the plurality of lines of text as selected text,the portion having a beginning location and an ending location; based atleast on user input indicating updates to the ending location of theselected text, wherein an initial line includes the ending locationprior to the updates and wherein a current line is initially the initialline, interactively adjusting the portion of the plurality of lines oftext displayed as the selected text while updates are received, by:updating the ending location along the current line of the text based ona position in the first direction indicated by the user input; andupdating the ending location based on a position in the second directionindicated by the user input, updating the ending location in the seconddirection comprising: while the computing device is operating in a firstmode and the user input indicates an updated ending location that isless than a first threshold distance away from the initial line in thesecond direction: maintaining the ending location in the current line;while the computing device is operating in the first mode and the userinput indicates an updated ending location that is greater than thefirst threshold distance away from the initial line in the seconddirection: changing the ending location to an adjacent line, theadjacent line being adjacent to the initial line and offset in thesecond direction, and entering a second mode after setting the currentline to be the adjacent line; and while the computing device isoperating in the second mode, using a second threshold distance in thesecond direction from the current line, wherein the second thresholddistance is less than the first threshold distance, to determine whetherto change the ending location of the selected text to a different linefrom the current line.
 6. The at least one storage device of claim 5,wherein the display screen is a touch screen, and wherein the methodfurther comprises: displaying a first gripper at a first gripperlocation associated with the beginning location; and displaying a secondgripper at a second gripper location associated with the endinglocation.
 7. The at least one storage device of claim 6, whereindisplaying the first gripper and the second gripper comprises:determining a width of the first gripper and the second gripper based ondisplay screen parameters.
 8. The at least one storage device of claim6, wherein: when the beginning location and the ending location arecloser together than a predetermined distance associated with a width ofthe first gripper and the second gripper, the second gripper location isa distance from the first gripper location such that the first gripperand the second gripper do not overlap.
 9. A computer having a connectionto provide output to a display screen that displays text comprising aplurality of lines of text, each line of text extending in a firstdirection and the plurality of lines of text being offset from oneanother in a second direction, the computer comprising: a processingsystem comprising at least one processing unit; at least one storagedevice having instructions stored thereon which, when executed by theprocessing system configures the processing system to: display a portionof the plurality of lines of text as selected text, the portion having abeginning location and an ending location; based at least on user inputindicating updates to the ending location of the selected text, whereinan initial line includes the ending location prior to the updates andwherein a current line is initially the initial line, interactivelyadjust the portion of the plurality of lines of text displayed as theselected text while updates are received, and wherein the processingsystem, to interactively adjust the portion of the plurality of lines,is further configured to: update the ending location along the currentline of the text based on a position in the first direction indicated bythe user input; and update the ending location based on a position inthe second direction indicated by the user input, and wherein to updatethe ending location in the second direction, the processing system isfurther configured to: while the processing system is operating in afirst mode and the user input indicates an updated ending location thatis less than a first threshold distance away from the initial line inthe second direction, maintain the ending location in the current line;while the processing system is operating in the first mode and the userinput indicates an updated ending location that is greater than thefirst threshold distance away from the initial line in the seconddirection: change the ending location to an adjacent line, the adjacentline being adjacent to the initial line and offset in the seconddirection, and enter a second mode after setting the current line to bethe adjacent line; and while the computing device is operating in thesecond mode, using a second threshold distance in the second directionfrom the current line, wherein the second threshold distance is lessthan the first threshold distance, to determine whether to change theending location of the selected text to a different line from thecurrent line.
 10. The computer of claim 9, wherein the display screen isa touch screen, and wherein the processing system is further configuredto: display a first gripper at a first gripper location associated withthe beginning location; and display a second gripper at a second gripperlocation associated with the ending location.
 11. The computer of claim10, wherein to display the first gripper and the second gripper theprocessing system is further configured to: determine a width of thefirst gripper and the second gripper based on display screen parameters.12. The computer of claim 10, wherein: when the beginning location andthe ending location are closer together than a predetermined distanceassociated with a width of the first gripper and the second gripper, thesecond gripper location is a distance from the first gripper locationsuch that the first gripper and the second gripper do not overlap.